The present invention relates generally to a manner by which to encode video data, such as a sequence of video frames of a video stream, which is to be communicated upon a bandwidth-limited communication channel. More particularly, the present invention relates to apparatus, and an associated method, by which to select the encoding rate at which to encode at least selected frames of the sequence. Selection is made to provide constant-average, bit-rate encoding, thereby to facilitate viewing of the frames of the video stream at client, or other video receiver, at a constant frame rate and a quality level of constant consistency.
Advancements in digital communication techniques have permitted the development of new and improved types of communications. Additional advancements shall permit continued improvements in communications and communication systems which make use of such advancements.
For instance, communication systems have been proposed, and implemented, for the communication of digital video data capable of forming video frames. Video images utilized during video conferencing are exemplary of applications which can advantageously make use of digital video sequences. Video streaming applications are also exemplary of applications which make use of digital video sequences.
A video frame is, however, typically formed of a large number of pixels, each of which is representable by a set of digital bits. And, a large number of video frames are typically required to represent any video sequence. Because of the large number of pixels per frame and the large number of frames required to form a typical video sequence, the amount of data required to represent the video sequence quickly becomes large.
Video sequences, like ordinary motion pictures recorded on film, comprise a sequence of still images, and the illusion of motion is created by displaying consecutive images at a relatively fast rate. For example, the display rate are between fifteen and thirty frames per second. Because of the relatively fast frame rate, the images in consecutive frames tend to be similar. A typical scene recorded by a camera comprises some stationary elements, such as, for example, background scenery and some moving parts. The moving parts may take many different forms, for example, the face of a news reader, moving traffic, and so on. Alternatively, the camera recording the scene may itself be moving, in which case all elements of the image have the same kind of motion. In many cases, this means that the overall change between one video frame and the next is rather small. Of course, this depends on the nature of the movement, the rate of the movement, i.e., the amount of change from one frame to the next.
The general similarity between consecutive frames of the video sequence permits the implementation of encoding techniques operable to reduce the amount of information which must be communicated between a video sending device and a video receiving device to permit the viewing of the video sequence at the video receiving device.
One exemplary encoding technique performs discrete cosine transforms of at least some pixels which collectively form each video frame. Encoding of the transformed values is performed, and the encoded values are sent by the video sending device to the video receiving device. As the communication channel upon which the encoded information is communicated to the video receiving device is generally bandwidth-limited, the encoding bit rate is not permitted to exceed an encoding rate which would exceed the communication capacity of the communication channel upon which the information is to be communicated. Additionally, when information is to be buffered at the video receiving device, the amount of information sent by the video sending device to the video receiving device also must not be at a rate greater than the rate at which the video receiving device is capable to buffer, and operate upon, the information.
Various manners, as a result, have been developed to encode video frames of a video sequence to facilitate sending of the video sequence upon a bandwidth-constrained communication channel to the video receiving device in a manner to permit viewing of the sequence at the video receiving device with at least a minimal viewing quality.
One technique, referred to as a TMN5 rate control technique, controls the bitrate of a video sequence encoded. The TMN5 rate control technique selectably skips video frames so that the amount of information encoded and sent upon the communication channel does not exceed the capacity of the communication channel or the video receiving device. While frame skipping reduces the amount of the average bit-rate encoding over a sequence of frames, skipping of selected video frames in a variable manner results in jerkiness of the resultant viewing of the sequence of the video frames at the video receiving device.
When the video sequence is to be communicated pursuant to a streaming video application, the video sequence need not, and typically is not, viewed in a real-time manner. Instead, the sequence is viewed only after at least a portion of the sequence is received at a video receiving device. If advantage could be taken of the nonreal-time nature of a video streaming application, or other application in which a sequence of video frames is communicated, the ability to provide a more constant frame rate and better quality-consistency of the video sequence, when viewed would be possible.
It is in light of this background information related to the communication of a sequence of video frames between a video sending device and a video receiving device that the significant improvements of the present invention have evolved.
The present invention, accordingly, advantageously provides apparatus, and an associated method, by which to encode video data, such as a sequence of video frames of a video stream, which is to be communicated upon a bandwidth-limited communication channel.
Through operation of an embodiment of the present invention, a manner is provided by which to select the encoding rate in which to encode at least selected frames of a sequence of video frames. A constant average bit-rate encoding of selected video frames of the sequence is performed, thereby to facilitate the viewing of the frames of the video stream at a client, or other video receiver, at a constant frame rate and at a consistent quality level.
In one aspect of the present invention, the constant average bit-rate at which selected frames of the sequence video are encoded is selected responsive to the channel capacity rate permitted of the communication channel upon which encoded information is to be communicated. Selection is made of the bit-rate at which video frames are to be encoded and which video frames are to be encoded. If appropriate, selection is also mode of which frames are to be discarded. Selection of which frames to encode and which frames to discard is made to facilitate viewing of the sequence of video frames, when received at the video receiving device, of improved quality-consistency and at a uniform frame rate.
In another aspect of the present invention, selection of the average bit-rate at which the selected video frames are encoded is further made responsive to indicia associated with a buffer located at a video receiving device. The buffer indicia is formed of, for instance, an indication of a time period required to buffer a selected amount of encoded information thereat. A model of the buffer is, for instance, maintained at the video sending device, and the buffer indicia is formed by the model of the buffer. The buffer models the buffering of information at the video receiving device by monitoring the information sent by the video sending device. Through use of a buffer model, feedback need not be sent by the receiving device back to the sending device.
In another aspect of the present invention, selection of the bit-rate by which to encode the selected ones of the video frames is further responsive to feedback information indicative of the encoding actually performed by an encoder of the video sending device. Feedback arrangement permits the selection of the bit rate at which to encode the selected video frames of the sequence to be increased or decreased, responsive to the feedback information to increase, or decrease, the selected encoding bit-rate, thereby to facilitate encoding operations performed by the encoder of the video sending device.
In one implementation, an encoding bit-rate is selected to form an encoding rate by which video frames of a streaming video sequence are to be encoded. A rate controller makes the selection. The rate controller is implemented, for instance, by execution of an algorithm at a processing device which selects the encoding bit-rate by which video frames of the streaming video sequence are encoded. Selection of the encoding bit rate is made responsive to a permitted communication channel rate, a target frame rate, and indicia associated with a buffer of a video receiving device at which encoded information is to be communicated. Responsive to the calculations, the encoding bit rate is selected, and selected video frames of the streaming video sequence are encoded and sent upon the communication channel to the video receiving device. Feedback information is also provided to the rate controller. The feedback information indications provided to the rate controller include the encoding bit rate at which the video frames are actually encoded. Responsive to the feedback information, selection of subsequent encoding bit rates are increased, or decreased, as appropriate, to maintain the encoding bit rate at the selected rate.
When encoding is performed at the encoding bit rate, and in the manners of embodiment of the present invention, improved quality-consistency and improved consistency of the frame viewing rate at a video receiving device is permitted. An improved viewing experience of the streaming video, or other sequence of video frames, is thereby provided.
In these and other aspects, therefore, apparatus, and an associated method, is provided for a video device operable in a communication system to send representations of a sequence of video frames upon a communication channel. A frame bit-rate encoding value is selected at which to encode at least selected ones of the video frames of the sequence of video frames. A rate controller is coupled to receive indications of a target frame rate representative of a targeted frame rate at which the representations of the videos frames of the sequence of video frames are to be communicated upon the communication channel and indications of a communication-channel capacity rate of permissible communication rates upon the communication channel. The rate controller is operable responsive thereto for determining the bit-rate encoding value at which to encode the at least selected ones of the video frames of the sequence. The frame bit-rate encoding value is of a level to permit the representation of the sequence to be communicated at the communication-channel capacity rate.
A more complete appreciation of the present invention and the scope thereof can be obtained from the accompanying drawings which are briefly summarized below, the following detailed description of the presently-preferred embodiments of the invention, and the appended claims.